In optimization-based approaches to inverse problems and to statistical estimation, it is common to augment the objective with a regularizer to address challenges associated with ill-posedness. The choice of a suitable regularizer is typically driven by prior domain information and computational considerations. Convex regularizers are attractive as they are endowed with certificates of optimality as well as the toolkit of convex analysis, but exhibit a computational scaling that makes them ill-suited beyond moderate-sized problem instances. On the other hand, nonconvex regularizers can often be deployed at scale, but do not enjoy the certification properties associated with convex regularizers. In this paper, we seek a systematic understanding of the power and the limitations of convex regularization by investigating the following questions: Given a distribution, what are the optimal regularizers, both convex and nonconvex, for data drawn from the distribution? What properties of a data source govern whether it is amenable to convex regularization? We address these questions for the class of continuous and positively homogenous regularizers for which convex and nonconvex regularizers correspond, respectively, to convex bodies and star bodies. By leveraging dual Brunn-Minkowski theory, we show that a radial function derived from a data distribution is the key quantity for identifying optimal regularizers and for assessing the amenability of a data source to convex regularization. Using tools such as $\Gamma$-convergence, we show that our results are robust in the sense that the optimal regularizers for a sample drawn from a distribution converge to their population counterparts as the sample size grows large. Finally, we give generalization guarantees that recover previous results for polyhedral regularizers (i.e., dictionary learning) and lead to new ones for semidefinite regularizers.
translated by 谷歌翻译
In this work, we give efficient algorithms for privately estimating a Gaussian distribution in both pure and approximate differential privacy (DP) models with optimal dependence on the dimension in the sample complexity. In the pure DP setting, we give an efficient algorithm that estimates an unknown $d$-dimensional Gaussian distribution up to an arbitrary tiny total variation error using $\widetilde{O}(d^2 \log \kappa)$ samples while tolerating a constant fraction of adversarial outliers. Here, $\kappa$ is the condition number of the target covariance matrix. The sample bound matches best non-private estimators in the dependence on the dimension (up to a polylogarithmic factor). We prove a new lower bound on differentially private covariance estimation to show that the dependence on the condition number $\kappa$ in the above sample bound is also tight. Prior to our work, only identifiability results (yielding inefficient super-polynomial time algorithms) were known for the problem. In the approximate DP setting, we give an efficient algorithm to estimate an unknown Gaussian distribution up to an arbitrarily tiny total variation error using $\widetilde{O}(d^2)$ samples while tolerating a constant fraction of adversarial outliers. Prior to our work, all efficient approximate DP algorithms incurred a super-quadratic sample cost or were not outlier-robust. For the special case of mean estimation, our algorithm achieves the optimal sample complexity of $\widetilde O(d)$, improving on a $\widetilde O(d^{1.5})$ bound from prior work. Our pure DP algorithm relies on a recursive private preconditioning subroutine that utilizes the recent work on private mean estimation [Hopkins et al., 2022]. Our approximate DP algorithms are based on a substantial upgrade of the method of stabilizing convex relaxations introduced in [Kothari et al., 2022].
translated by 谷歌翻译
Prototyping and validating hardware-software components, sub-systems and systems within the intelligent transportation system-of-systems framework requires a modular yet flexible and open-access ecosystem. This work presents our attempt towards developing such a comprehensive research and education ecosystem, called AutoDRIVE, for synergistically prototyping, simulating and deploying cyber-physical solutions pertaining to autonomous driving as well as smart city management. AutoDRIVE features both software as well as hardware-in-the-loop testing interfaces with openly accessible scaled vehicle and infrastructure components. The ecosystem is compatible with a variety of development frameworks, and supports both single and multi-agent paradigms through local as well as distributed computing. Most critically, AutoDRIVE is intended to be modularly expandable to explore emergent technologies, and this work highlights various complementary features and capabilities of the proposed ecosystem by demonstrating four such deployment use-cases: (i) autonomous parking using probabilistic robotics approach for mapping, localization, path planning and control; (ii) behavioral cloning using computer vision and deep imitation learning; (iii) intersection traversal using vehicle-to-vehicle communication and deep reinforcement learning; and (iv) smart city management using vehicle-to-infrastructure communication and internet-of-things.
translated by 谷歌翻译
Neural networks have revolutionized the area of artificial intelligence and introduced transformative applications to almost every scientific field and industry. However, this success comes at a great price; the energy requirements for training advanced models are unsustainable. One promising way to address this pressing issue is by developing low-energy neuromorphic hardware that directly supports the algorithm's requirements. The intrinsic non-volatility, non-linearity, and memory of spintronic devices make them appealing candidates for neuromorphic devices. Here we focus on the reservoir computing paradigm, a recurrent network with a simple training algorithm suitable for computation with spintronic devices since they can provide the properties of non-linearity and memory. We review technologies and methods for developing neuromorphic spintronic devices and conclude with critical open issues to address before such devices become widely used.
translated by 谷歌翻译
The number of malware is constantly on the rise. Though most new malware are modifications of existing ones, their sheer number is quite overwhelming. In this paper, we present a novel system to visualize and map millions of malware to points in a 2-dimensional (2D) spatial grid. This enables visualizing relationships within large malware datasets that can be used to develop triage solutions to screen different malware rapidly and provide situational awareness. Our approach links two visualizations within an interactive display. Our first view is a spatial point-based visualization of similarity among the samples based on a reduced dimensional projection of binary feature representations of malware. Our second spatial grid-based view provides a better insight into similarities and differences between selected malware samples in terms of the binary-based visual representations they share. We also provide a case study where the effect of packing on the malware data is correlated with the complexity of the packing algorithm.
translated by 谷歌翻译
尽管机器人学课程在高等教育方面已建立,但这些课程通常专注于理论,有时缺乏对开发,部署和将软件应用于真实硬件的技术的系统覆盖。此外,大多数用于机器人教学的硬件平台是针对中学水平的年轻学生的低级玩具。为了解决这一差距,开发了一个自动驾驶汽车硬件平台,称为第1 f1 f1tth,用于教授自动驾驶系统。本文介绍了以“赛车”和替换考试的竞赛为主题的各种教育水平教学模块和软件堆栈。第1辆车提供了一个模块化硬件平台及其相关软件,用于教授自动驾驶算法的基础知识。从基本的反应方法到高级计划算法,教学模块通过使用第1辆车的自动驾驶来增强学生的计算思维。第1辆汽车填补了研究平台和低端玩具车之间的空白,并提供了学习自主系统中主题的动手经验。多年的四所大学为他们的学期本科和研究生课程采用了教学模块。学生反馈用于分析第1个平台的有效性。超过80%的学生强烈同意,硬件平台和模块大大激发了他们的学习,而超过70%的学生强烈同意,硬件增强了他们对学科的理解。调查结果表明,超过80%的学生强烈同意竞争激励他们参加课程。
translated by 谷歌翻译
在电缆驱动的平行机器人(CDPR)中,单个电缆故障通常会导致整个机器人的完全故障。但是,通常可以通过重新配置框架上的电缆附件来恢复丢失的静态工作空间(由于故障)。通过将运动冗余以在实时冗余分辨率控制器中操纵的移动线性滑块的形式添加到机器人中,从而引入了此功能。提出的工作将该控制器与在线故障检测框架相结合,以开发自动任务恢复的完整失误耐受控制方案。该解决方案通过将最终效应器的姿势估计与仅依靠最终效应器信息的交互式多重模型(IMM)算法相结合,从而提供了鲁棒性。然后将故障和姿势估计方案绑定到冗余分辨率方法中,以产生无缝的自动任务(轨迹)恢复方法,以实现电缆故障。
translated by 谷歌翻译
给定尺寸$ d $中的独立标准高斯点$ v_1,\ ldots,v_n $,对于$(n,d)$的值(n,d)$的值很高,概率很高,同时通过所有要点?将椭圆形拟合到随机点的基本问题与低级别矩阵分解,独立的组件分析和主成分分析有连接。基于有力的数值证据,桑德森,帕里洛和威尔斯基[Proc。关于决策和控制会议,第6031-6036页,2013年]猜想,椭圆形拟合问题的问题从可行的到不可行的$ n $增加,并在$ n \ sim d^2/4处急剧阈值$。我们通过为某些$ n = \ omega(\,d^2/\ log^5(d)\,)$构建合适的椭圆形来解决这个猜想,从而改善了Ghosh等人的先前工作。 [Proc。关于计算机科学基础的研讨会,第954-965、2020页],需要$ n = o(d^{3/2})$。我们的证明证明了Saunderson等人的最小二乘结构的可行性。使用对特定非标准随机矩阵的特征向量和特征值进行仔细的分析。
translated by 谷歌翻译
学习证明(POL)建议模型所有者使用机器学习培训检查站,以建立已经花费了必要的培训计算的证明。 POL FIREGO加密方法和贸易严格的安全性的作者通过适用于随机梯度下降和适应性变体,可扩展到深度学习。缺乏正式分析使攻击者可能能够为他们没有训练的模型提供证据。我们对为什么不能正式(DIS)正式分析POL协议可抵抗欺骗对手。为此,我们在POL中解开了证明验证的两个角色:(a)有效确定证明是否是有效的梯度下降轨迹,以及(b)确定优先级,使在培训完成后制作证明(即。 ,欺骗)。我们表明,有效的验证会导致接受合法证明和拒绝无效的证据之间的权衡,因为深度学习必然涉及噪音。没有针对这种噪声如何影响训练的精确分析模型,我们无法正式保证POL验证算法是否强大。然后,我们证明,建立优先级也可以鲁棒化地减少到学习理论中的一个开放问题:欺骗Pol Pol hoc hoc训练类似于在非凸X学习中找到具有相同终点的不同轨迹。但是,我们不严格地知道对最终模型权重的先验知识是否有助于发现此类轨迹。我们得出的结论是,在解决上述开放问题之前,可能需要更严重地依靠密码学来制定新的POL协议,并提供正式的鲁棒性保证。特别是,这将有助于建立优先级。作为我们分析的见解的副产品,我们还展示了对POL的两次新攻击。
translated by 谷歌翻译
我们考虑主人想要在$ n $ Workers上运行分布式随机梯度下降(SGD)算法的设置,每个算法都有一个数据子集。分布式SGD可能会遭受散乱者的影响,即导致延迟的缓慢或反应迟钝的工人。文献中研究的一种解决方案是在更新模型之前等待每次迭代的最快$ k <n $工人的响应,其中$ k $是固定的参数。 $ k $的价值的选择提供了SGD的运行时(即收敛率)与模型错误之间的权衡。为了优化误差折衷,我们研究了在整个算法的运行时,以自适应〜$ k $(即不同的$ k $)调查分布式SGD。我们首先设计了一种自适应策略,用于改变$ k $,该策略根据我们得出的墙壁通行时间的函数,基于上限的上限来优化这种权衡。然后,我们建议并实施一种基于统计启发式的自适应分布式SGD的算法。我们的结果表明,与非自适应实现相比,分布式SGD的自适应版本可以在更少的时间内达到较低的误差值。此外,结果还表明,自适应版本是沟通效率的,其中主人与工人之间所需的通信量小于非自适应版本的沟通量。
translated by 谷歌翻译